Barrier Synchronization Pattern

Parallel algorithms divide the work into multiple, concurrent tasks. These tasks or UEs may execute in parallel depending on the physical resources available. It is common for UEs to proceed in phases where the next phase cannot start until all UEs complete the previous phase. This is typically due to mutual dependency on the data written during the previous phase by concurrent UEs. Since UEs may execute at different speeds, there is a need for UEs to wait for one another before proceeding to the next phase. Barriers are commonly used to enforce such waiting. Figure 1 illustrates how a barrier works. A UE executes its code until it reaches a barrier. Then it waits until all other UEs have reached that barrier before proceeding. Consider the Barnes-Hut [BH86] N-body simulation algorithm. This is an iterative algorithm with well-defined phases: building the octree, calculating the forces between bodies, updating the positions and velocities of each body. One way to parallelize the algorithm is to have multiple UEs perform the three different phases. However, no UE can proceed to the next phase until all UEs complete executing the previous phase. After all, it does not make sense to update the position when some UEs are still calculating the forces between bodies. A barrier where all UEs wait for each other to reach the barrier before continuing with their respective computation, is called a global barrier. We distinguish a global barrier from another kind of barrier called local barrier, where a parent task waits for all the child tasks to finish before it can continue.